A critical evaluation of the algorithm behind the Relative Citation Ratio (RCR)

The influence of scientific publications is increasingly assessed using quantitative approaches, but most available metrics have limitations that hamper their utility [1]. Hutchins and colleagues recently proposed the Relative Citation Ratio (RCR) [2], which compares the citation rate of an article against the citation rate that is expected for its field. The metric is an attractive and intuitive solution to indicate whether an article is cited more or less frequently than its “peer” publications. As a ratio of rates, RCR is an article-level metric that is field and time independent and strongly correlated with peer review evaluations [2]; the metric was central in the proposed (and withdrawn) grant management policy of the National Institutes of Health (NIH) [3] even though the RCR has been criticized for lacking a theoretical model, having insufficient transparency, and having poor correlation with other peer evaluations [4,5]. We analyzed the algorithm behind the RCR and report several concerns about the calculation of the metric that may limit its utility.